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NEURONAL  MECHANISMS  OF  INTELLIGENCE 


I.  I  SUMMARY 

The  aim  of  this  research  program  was  to  identify  the  functional  unit  in  the 
brain  for  reward  or  positive  reinforcement.  On  the  assumption  that  the  simplest 
possible  unit  is  the  single  brain  cell,  we  have  attempted  to  reinforce  individual 
neuronal  firing  patterns  by  direct  applications  of  neuro transmitters  or  drugs  to  the 
cell  soma.  Our  most  satisfactory  experiments  have  been  performed  on  large 
pyramidal  cells  in  hippocampal  brain  slices.  The  probability  of  neuronal  firing 
increased  sharply  when  reinforced  by  contingent  applications  of  dopamine  or 
cocaine;  the  same  injections  applied  independently  of  neuronal  firing  had  no  such 
effect  and  in  fact  tended  to  suppress  activity.  There  is  an  indication  of 
pharmacological  specificity:  included  among  substances  that  are  ineffective  are 
GABA,  serotonin,  acetylcholine,  imipramine,  ethanol,  and  saline.  Some  features 
of  behavioral  operant  conditioning  are  not  observed  in  the  neuronal  experiments; 
reinforcement  schedules  are  ineffective  and  relearning  is  not  enhanced.  Such 
features  thus  may  reflect  properties  of  neuronal  systems  rather  than  of  individual 
cells.  Finally,  we  have  begun  to  consider  the  biochemical  events  that  may  mediate 
the  cellular  reinforcement  process.  Proteins  that  control  cellular  firing  rates  may 
be  modified  (phosphorylated)  via  a  biochemical  cascade  involving  the  conjunction 
of  influx  and  dopamine  receptor  stimulation.  '  C^\  or  v. 
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H.  INTRODUCTION 

Our  overall  aim  is  to  understand  how  the  brain  produces  intelligent  behavior. 
What  are  the  essential  properties  and  functions  of  brain  cells,  cellular  and 
organizational,  on  which  intelligence  depends?  The  present  research  is  based  on 
the  premise  that  human  intelligence  has  evolved  from  the  goal-seeking  brain 
functions  of  lower  forms,  and  that  these  functions  in  turn  depend  on  a  capacity  for 
behavior  to  be  strengthened  or  positively  reinforced  by  certain  stimulus 
consequences  or  rewards.  Our  immediate  aim  is  to  identify  and  characterize  the 
smallest  functional  unit  in  the  brain  for  reward  or  positive  reinforcement. 
Knowledge  of  this  presumed  unit  of  organization  of  biological  intelligence  could 
then  be  used  to  define  a  suitable  unit  of  organization  of  machine  intelligence. 

The  phenomenon  of  positive  reinforcement  is  well  established  at  the  level  of 
the  whole  animal.  If  a  response  is  closely  and  regularly  followed  by  a  reinforcing 
stimulus,  the  behavior  is  strengthened  or  its  probability  is  increased.  A  behavioral 
response  obviously  reflects  the  activity  of  many  neurons.  Is  it  the  integrated 
activity  of  these  neurons  that  is  reinforced;  that  is,  is  reinforcement  exerted  at  the 
level  of  neuronal  systems?  Or  is  it  the  individual  activities  of  the  relevant  neurons 
that  is  reinforced;  that  is,  is  reinforcement  exerted  at  the  cellular  level  (Klopf, 
1982)? 

It  is  commonly  believed  that  reinforcement  is  exerted  at  the  systems  level. 
According  to  this  view,  the  reinforcement  of  behavioral  responses  is  paralleled  at 
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the  neuronal  level  by  the  strengthening  or  reorganization  of  complex  neuronal 
circuits  or  assemblies  (Hebb,  1949).  Unfortunately,  no  such  circuit  or  cell  assembly 
for  any  reinforced  behavior  has  yet  been  identified.  Where  in  the  rat's  brain,  for 
example,  would  one  find  the  circuit  for  a  lever-press  response?  And  how  would  one 
measure  the  changes  in  circuitry  that  are  induced  when  lever-pressing  is 
reinforced?  These  anatomical  and  physiological  considerations,  as  well  as 
difficulties  at  the  behavioral  level  associated  with  response  definition,  have  led 
some  psychologists  to  ask  whether  it  really  is  correct  to  say  that  the  whole 
response  is  the  functional  unit  for  reinforcement.  According  to  Skinner  (1953),  a 
more  useful  conceptual  scheme  assumes  that  all  responses  are  made  up  of 
dements,  and  that  it  is  these  elements  or  "behavioral  atoms"  (p.  94),  and  not  whole 
responses,  that  are  the  units  strengthened  by  reinforcement.  If  so,  and  if  atoms  of 
behavior  can  be  represented  by  the  activity  of  individual  neurons,  then  positive 
reinforcement  may  actually  occur  at  the  level  of  individual  cells,  as  suggested  by 
Klopf  (1982). 

The  cellular  hypothesis  suggests  an  analogy  between  reinforced  behavior  and 
an  ant  colony.  If  you  watch  an  ant  colony  from  a  distance  as  it  descends  on  a  pile 
of  sugar,  it  looks  much  like  one  large  creature  moving  toward*  the  sugar. 
However,  dose  up,  the  colony  is  seen  to  be  no  single  creature,  but  rather  an 
aggregated  mass  of  individual  ants.  In  the  terms  of  the  analogy,  the  important 
question  for  a  theory  of  reinforcement  is,  who  tastes  the  sugar?  Obviously  it  is  not 
the  ant  colony  as  a  whole,  but  the  individual  ants  themselves  that  actually  taste 
the  sugar.  Similarly,  it  could  be  argued  that  in  the  brain  it  is  the  behavior  of 
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individual  neurons,  and  not  the  integrated  activity  of  neuronal  circuits,  that  is 
directly  modified  by  reinforcing  signals. 
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ID.  REFLEXES  AND  OPERANTS 

According  to  Skinner  (1938),  environmental  stimuli  exert  control  over 
behavior  by  two  fundamentally  different  processes — respondent  and  operant.  The 
distinction  is  based  entirely  on  the  temporal  relationship  between  the  controlling 
stimulus  and  the  response  (Table  1).  In  the  case  of  respondent  (reflex)  processes, 
the  controlling  stimulus  precedes  the  response  and  elicits  it,  as,  for  example,  food 
in  the  mouth  elicits  salivation.  In  the  case  of  operant  processes,  the  controlling 
stimulus  follows  the  response  and  reinforces  it,  as,  for  example,  food  following  a 
lever-press  response  makes  lever  pressing  more  probable. 

Operant  or  reinforced  behavior  thus  is  controlled  by  its  consequences,  which 
change  the  likelihood  of  future  occurrences  of  the  behavior.  Consequences  that 
increase  response  probabilities  are  termed  positive  reinforcers.  Only  the  temporal 
relationship  of  stimulus  and  response,  and  not  the  nature  of  the  response  itself, 
distinguishes  respondent  from  operant  behavior.  For  example,  blinking  elicited  by 
a  cinder  in  the  eye  is  respondent,  whereas  the  same  eyelid  closure,  winking  to 
attract  a  member  of  the  opposite  sex,  is  operant. 

From  an  evolutionary  point  of  view,  the  emergence  of  operant  behavior  had 
profound  implications.  With  only  a  reflex  brain,  an  organism  is  essentially  reactive 
and  controlled  by  impinging  stimulation.  Through  Pavlovian  conditioning,  responses 
could  come  under  the  control  of  new  stimuli  as  conditioned  reflexes,  but  the 


Type  of  Process 

Controlling 

Stimulus 

Effect  on 
Behavior 

Hypothesized 
Linkage  of 
Stimulus  (S) 
and  Response  (R) 

Hypothesized 
Locus  of 
Conditioning 

Classical 

Antecedent 

Elicitation 

S-R 

Synapse 

Operant 

Consequent 

Reinforcement 

R-S 

Intracellular 
pacemaker 
and/or  Synapse 

organism  remains  stimulus  bound.  As  Skinner  (1981)  points  out,  operant  condition¬ 
ing  must  have  evolved  in  parallel  with  two  other  products  of  natural  selection  —  "a 
susceptibility  to  reinforcement  by  certain  kinds  of  consequences  and  a  supply  of 
behavior  ...which  has  little  or  no  relation  to  (eliciting  or  releasing)  stimuli."  (p. 
501).  As  a  consequence,  the  organism  with  operant  capability  is  not  stimulus  bound 
but  intrinsically  active,  and  can  learn  to  find  more  favorable  stimulation  than  that 
provided  by  its  current  environment. 

It  is  interesting  to  consider  whether  or  not  fundamental  changes  in  brain 
physiology  were  required  for  the  evolution  of  operant  behavior.  Respondent 
mechanisms  derive  directly  from  classical  Sherringtonian  physiology— a  specific 
stimulus  elicits  a  particular  response  by  activation  of  a  reflex  arc.  Operant 
processes,  on  the  other  hand,  are  not  obviously  compatible  with  classical 
physiology.  Impulses  in  a  reflex  arc  are  conducted  in  an  afferent  to  efferent 
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direction;  hence,  the  neuronal  circuitry  requires  that  the  stimulus  precede  the 
response  (S-R).  But  operants  are  controlled  by  stimuli  that  follow  the  response. 
Since  the  reinforcing  stimulus  occurs  only  after  the  response  has  been  completed, 
operant  behavior  would  seem  to  require  an  unconventional  circuitry  in  which 
efferents  are  activated  before  afferents  (R-S). 

The  classical  solution  to  this  problem  has  been  to  assume  that  all  responses 
(operants  as  well  as  respondents)  are  elicited  by  antecedent  stimuli,  and  that  the 
reinforcing  stimulus  acts  merely  to  strengthen  the  connectivity  or  linkage  between 
these  eliciting  stimuli  and  the  reinforced  response  (Thorndike,  1911;  Hull,  1943). 
According  to  this  view,  operant  responses  are  little  more  or  less  than  Pavlovian 
conditioned  reflexes;  as  such,  operants  can  obviously  be  mediated  by  reflex  arcs, 
and  both  operant  and  classical  conditioning  can  be  explained  at  the  physiological 
level  by  the  same  fundamental  mechanisms  of  synaptic  change. 

Skinner  (1953)  has  criticized  this  view  on  the  grounds  that  it  is  probably 
impossible  to  show  that  any  single  stimulus  invariably  precedes  an  operant 
response.  (What  stimulus,  for  example,  will  reliably  elicit  a  lever-press  response 
from  an  untrained  rat?)  In  the  absence  of  eliciting  stimuli,  operants  must  be 
generated  intrinsically~in  Skinner's  terminology,  operants  are  said  to  be  "emitted" 
rather  than  elicited.  Furthermore,  any  incidental  stimulation  that  may  have 
occurred  prior  to  the  operant  response  has  no  necessary  relationship  to  future 
reinforcement.  In  the  case  of  operants,  only  responses  have  fixed  relationships  to 
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reinforcement.  Therefore,  the  neural  connections  important  for  operant  behavior 
must  be  those  between  responses  and  reinforcing  stimuli  (or,  more  precisely, 
between  the  brain  elements  that  represent  responses  and  those  that  represent 
reinforcers).  Thus,  if  the  classical  view  conceives  all  behavior  to  be  mediated  by 
S-R  connections,  Skinner's  view  conceives  only  respondents  to  be  mediated  by  S-R 
connections  and  requires,  for  the  mediation  of  operants,  a  new  functional  unit  that 
can  act  in  an  R-S  mode.  This  new  unit  will  be  designated  the  "operant  element". 


-8- 
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IV.  NEURONAL  MODELS  OF  OPERANT  REINFORCEMENT 

The  operant  element  proposed  here  resembles  in  many  ways  the  "elementary 
heterostat"  posited  in  Klopfs  (1982)  theory  of  intelligent  systems,  although,  as  we 
shall  see,  the  heterostat  functions  in  the  classical,  Thorndikean  S-R  mode  rather 
than  in  a  Skinnerian  R-S  mode.  Both  units  represent  neuronal  models  of  operant 
conditioning,  and  both  are  functional  elements  intended  to  serve  as  fundamental 
building  blocks  for  complex  behavior.  In  both  cases,  operant  conditioning  is 
conceived  as  an  adaptive  change  in  neuronal  activity  induced  by  reinforcement 
signals  received  after  the  neuron  has  discharged. 

Klopf  (1982)  has  put  his  idea  vividly  and  succinctly:  "After  a  neuron  fires,  it 
waits  for  a  few  hundred  milliseconds  or  more  to  see  how  it  will  be  affected  by  the 
action  it  has  taken.  If  it  experiences  further  depolarization  within  a  second  or  so, 
it  increases  the  effectiveness  of  the  excitatory  synapses  that  led  to  its  firing  in  the 
first  place,  thereby  increasing  the  probability  that  it  will  fire  the  next  time  that 
some  fraction  of  these  synapses  is  active"  (p.  5).  Klopfs  heterostat  is  diagrammed 
as  a  3-neuron  assembly  in  Fig.  1,  which  shows  a  "reinforced"  neuron  and  its 
requisite  synaptic  connections  with  distinct,  but  chemically  unspecified,  excitatory 
"input"  and  "reinforcement"  neurons. 

The  heterostat  may  be  compared  with  the  operant  element  of  the  present 
proposal,  diagrammed  as  a  2-neuron  assembly  in  Fig.  1.  Again,  we  have  a  neuron 


-9- 
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A. HETEROSTAT  B. OPERANT  ELEMENT 


Neuronal  models  of  operant  conditioning.  (A)  Diagram  of  Klopfs 
(1982)  elementary  heterostat  as  a  3-neuron  assembly,  with 
"input",  "reinforced",  and  "reinforcing"  neurons  which  subserve 
stimulus,  response,  and  reinforcement  functions,  respectively. 
Sequential  firing  of  "input",  "reinforced"  and  "reinforcing"  neurons 
in  appropriate  temporal  relation  increases  the  effectiveness  of 
the  excitatory  synapse  between  the  "input"  neuron  and  the 
"reinforced"  neuron.  (B)  Diagram  of  operant  element.  Only  two 
neurons  are  required:  a  "reinforced"  or  command  neuron  capable 
of  spontaneous  activity  and  a  catecholamine-  or  endorphin¬ 
releasing  neuron  which  is  chemically  specialized  for  "reinforcing" 
functions.  When  the  spontaneous  activity  of  the  command  neuron 
is  regularly  followed  by  activation  of  its  catecholamine  or 
endorphin  receptors  (due  to  contingent  firing  of  the  "reinforcing" 
neuron),  the  intracellular  pacemaker  of  the  command  neuron  is 
activated.  See  text  for  further  explanation. 


that  receives  reinforcement  signals  from  a  "reinforcement"  neuron,  but  the 
reinforcement  neuron  is  chemically  specialized  and  the  "reinforced"  neuron  (which 
we  will  now  call  a  command  neuron)  must  exhibit  intrinsically  generated  spike 
activity,  even  in  the  absence  of  excitatory  synaptic  inputs.  The  operation  of  the 
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operant  element  is  relatively  simples  if  the  spontaneous  firing  of  a  command 
neuron  is  shortly  followed  by  activation  of  its  chemically  specialized  reinforcement 
receptors  (due  to  release  of  endorphins  or  catecholamines  from  a  reinforcement 
neuron),  the  intracellular  pacemaker  of  the  command  neuron  will  be  energized  and 
its  spontaneous  activity  will  therefore  be  increased.  On  the  other  hand,  if 
reinforcement  receptors  are  activated  during  periods  of  silence,  the  intracellular 
pacemaker  will  not  be  energized  and  spontaneous  spike  activity  will  not  be 
affected  or  may  even  be  reduced.  Finally,  if  the  augmented  spike  activity  of  a 
previously  reinforced  command  neuron  is  no  longer  followed  by  reinforcement,  the 
intracellular  pacemaker  will  be  gradually  deactivated  or  extinguished  and 
spontaneous  firing  will  return  to  a  baseline  level. 

Keeping  in  mind  the  many  points  of  similarity,  it  is  instructive  to  contrast 
the  heterostat  and  the  operant  element.  First,  the  two  models  differ  with  respect 
to  the  nature  and  locus  of  the  reinforcing  effect.  In  the  case  of  the  heterostat, 
operant  reinforcement  is  assumed  to  act  at  the  synapse  and  to  increase  excitatory 
conductances;  in  the  case  of  the  operant  element,  reinforcement  is  assumed  to  act 
intracellularly  and  to  energize  an  action  potential  generator  or  pacemaker.  The 
two  models  also  differ  with  respect  to  the  sign  and  chemical  specificity  of  the 
reinforcement  message.  The  heterostat  requires  depolarizing  reinforcement 
signals,  but  their  chemical  nature  is  not  important.  The  operant  element  requires 
chemically  specified  reward  messages,  but  the  sign  of  the  postsynaotic  potential  is 
not  important.  Indeed,  it  is  interesting  that  the  enkephalin  and  catecholamine 


Stein,  L.  and  Belluzzi,  J.D. 


Final  Technical  Report  F49620-81-K-0015 


reinforcement  transmitters  proposed  in  the  operant-element  model  generally  exert 
acute  hyperpolarizing  effects  in  neurophysiological  experiments.  Whether  such 
inhibitory  actions  are  a  necessary  characteristic  of  reinforcement  transmitters 
remains  to  be  determined.  In  any  case,  it  seems  clear  that  the  immediate  synaptic 
effects  of  reinforcement  transmitters,  which  often  inhibit  cellular  activity,  must 
be  distinguished  from  their  longer-term  reinforcing  actions,  which  facilitate 
cellular  activity. 
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V.  NEUROCHEMICAL  SUBSTRATES  OF  REINFORCEMENT 

An  important  role  for  catecholamines  and  endorphins  in  the  reinforcement 
process  is  suggested  by  evidence  from  self-stimulation  and  self-administration 
experiments  (for  reviews  see  Crow,  1973;  Fibiger,  1978;  Stein,  1978;  Stein  3c 
Belluzzi,  1979;  Wise  3c  Bozarth,  1982).  In  the  self-stimulation  experiments, 
electrodes  were  implanted  in  brain  pathways  which  release  catecholamines  or 
endorphins  upon  stimulation.  Although  the  sole  reward  for  operant  behavior  was  an 
electrical  brain  stimulus,  very  high  response  rates  were  observed  (Olds  3c  Milner, 
1954;  Stein,  1964;  Crow,  1972;  Belluzzi  3c  Stein,  1977).  Anatomical  mapping  of 
positive  and  negative  sites  was  generally  consistent  with  the  suggestion  that 
catecholamines  and  endorphins  were  involved  with  reinforcement.  This  idea  was 
further  supported  by  pharmacological  experiments.  Antagonists  of  catecholamines 
and  endorDhins,  such  as  chlorpromazine  and  naloxone,  would  be  expected  to  block 
chemical  transmission  of  reinforcement  messages.  In  support  of  the  model,  these 
drugs  blocked  self-stimulation  behavior  (Olds,  1962;  Holtzman,  1976;  Belluzzi  3c 
Stein,  1977). 

In  the  self-administration  experiments,  the  sole  reinforcement  for  operant 
behavior  was  central  or  systemic  injections  of  endorphins  (Belluzzi  3c  Stein,  1977; 
Mello  3c  Mendelson,  1979;  Olds  3c  Williams,  1980)  or  dopamine-like  compounds  such 
as  apomorphine  and  piribedil  (Baxter  et  al.,  1974;  Yokel  3c  Wise,  1978).  Animals 
work  avidly  for  the  chemical  injections,  just  as  they  do  for  electrical  stimulation. 
The  patterns  of  self-administration  often  resemble  the  patterns  observed  when 
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cocaine,  amphetamine  or  morphine  are  self-administered  (Baxter  et  al.,  19741.  In 
addition,  drugs  such  as  chlorpromazine,  which  block  the  reinforcing  effects  of 
cocaine  and  amphetamine,  also  block  apomorphine  self-administration  (Baxter  et 
al.,  1976;  Wise,  1978). 

Although  these  anatomical  and  pharmacological  findings  seem  consistent  with 
the  idea  that  reinforcing  functions  are  performed  by  specialized  systems  of 
catecholamine  and  endorphin  neurons,  major  problems  remain.  The  anatomical 
data  are  primarily  correlational  and  insufficiently  detailed;  it  has  not  yet  been 
shown  that  any  map  of  positive  self-stimulation  sites  overlaps  precisely  the  map  of 
catecholamine  or  endorphin  pathways  in  that  region.  Furthermore,  effective  fields 
of  stimulation  are  so  large  relative  to  the  size  of  the  units  stimulated  that 
self-stimulation  electrodes  must  cause  the  simultaneous  release  of  many  different 
transmitters,  including  some  that  are  still  unknown.  At  the  same  time,  almost  all 
of  the  pharmacological  reports  can  be  criticized  because  it  is  difficult  to 
distinguish  specific  drug  effects  on  reinforcement  from  nonspecific  effects  on 
performance  (Phillips  <5c  Fibiger,  1978;  Wise,  1978).  In  addition,  there  are  findings 
which  seem  to  directly  contradict  the  norepinephrine  and  dopamine  theories  of 
reinforcement.  Lesions  of  the  dorsal  noradrenergic  bundle  failed  to  eliminate  locus 
coeruleus  self-stimulation  (Clavier  <5c  Routtenberg,  1976)  and  lesions  of  the 
nigro-striatal  bundle  failed  to  eliminate  substantia  nigra  self-stimulation  (Clavier 
&  Fibiger,  1977).  Self-administration  studies  generally  provide  a  stronger  line  of 
evidence  for  catecholamine  involvement  in  reinforcement  processes,  but 
difficulties  have  been  raised  by  conflicting  findings  with  the  norepinephrine  agonist 
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donidine  (Davis  Sc  Smith,  1977;  Yokel  <5 c  Wise,  1978)  and  the  paradoxical 
observation  that  the  dopamine  antagonist  haloperidol  supports  self-administration 
(Glick  Sc  Cox,  1975). 

Such  difficulties  have  led  to  the  suggestion  that  exclusive  reliance  on 
conventional  self-stimulation  and  self-administration  methods  is  inappropriate  for 
the  study  of  reinforcement  mechanisms,  and  that  new  and  more  powerful  methods 
must  be  developed  (Hall,  Bloom  Sc  Olds,  1977).  It  is  possible  that  a  simpler 
preDaration  than  the  whole  animal  may  be  required  to  provide  definitive  tests  of 
the  catecholamine-endorphin  reinforcement  theory.  Since  the  simplest  biological 
system  whose  behavior  may  be  capable  of  reinforcement  is  the  single  cell,  a  second 
objective  of  the  present  research  was  to  provide  more  definitive  tests  of  the 
catecholamine- endorphin  reinforcement  theory. 


-15- 


Stein,  L.  and  Belluzzi,  J.D. 


Final  Technical  Report  F49620-81-K-0015 


VI.  OPERANT  CONDITIONING  OF  SINGLE  NEURONS 

Olds  (1965)  was  the  first  to  report  apparent  evidence  for  the  operant 
conditioning  of  single  neurons.  In  these  experiments,  rats  with  implanted 
microelectrodes  received  food  or  rewarding  brain  stimulation  contingent  on 
appropriate  bursts  of  single  unit  activity.  Firing  rates  were  increased  in  a  number 
of  cases,  suggesting  reinforcement  of  the  single  unit  response.  Unfortunately,  it  is 
not  clear  whether  it  was  the  behavior  of  the  individual  neuron  that  was  being 
reinforced  or  whether  some  more  complex  response  or  movement,  of  which  the 
neuron's  activity  was  a  part,  actually  was  being  reinforced.  In  some  of  Olds'  tests  a 
restriction  system  was  used  to  limit  movement:  electronic  detectors  were 
discharged  by  most  movements  and  these  precluded  reinforcement.  Although 
operant  conditioning  was  still  obtained  under  these  conditions,  one  cannot  rule  out 
the  possible  reinforcement  of  behaviors  involving  undetected  movements,  such  as 
postural  adjustments  or  attentional  responses.  Like  other  investigators  who  have 
attempted  to  demonstrate  operant  conditioning  of  single  unit  activity  (Fetz,  1969; 
Wyler  <3e  Robbins,  1980),  Olds  recognized  that,  if  a  reinforcing  stimulus  is  delivered 
to  a  behaving  animal,  it  is  impossible  to  separate  the  reinforcement  of  single  units 
from  the  reinforcement  of  more  complex  responses. 
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release  of  catecholamines  or  endorphins.  If  so,  it  might  be  possible  to  duplicate 
these  natural  signals  of  reinforcement  by  the  direct  application  of  the  transmitters 
to  appropriate  neurons.  In  such  experiments  catecholamines  or  endorphins  would 
be  applied  via  micropipettes  to  individual  units  in  various  regions  of  the  brain 
following  particular  rates  or  patterns  of  activity.  These  rates  or  patterns  should  be 
strengthened  only  by  contingent  application  of  the  transmitters,  and  not  by 
noncontingent  application. 

As  indicated  above,  the  neuronal  operant  conditioning  procedure  could 
provide  more  definitive  answers  to  a  number  of  basic  questions  concerning  the 
neurochemical  nature  of  the  reinforcement  substrate.  For  example,  the  relative 
efficacy  of  dopamine  and  norepinephrine  as  reinforcers  could  be  assessed  by  means 
of  the  neuronal  operant  conditioning  method.  And  if  both  catecholamines  proved 
effective,  their  sites  of  reinforcing  action  could  be  determined  by  testing  units  in 
various  locations.  In  the  same  way,  the  question  of  which  opiate  peptide  may  be 
specialized  for  reinforcing  functions  could  be  answered  by  determining  the  relative 
reinforcing  potency  of  the  different  endorphins.  Similarly,  identification  of  the 
precise  opiate  receptor  that  might  be  specialized  for  reinforcement  functions  could 
be  assessed  by  use  of  highly  specific  opiate  receptor  agonists.  It  would  also  be 
possible  to  provide  information  about  the  possible  interactions  of  catecholamine 
and  endorphin  transmitters  in  the  mediation  of  reinforcement  functions.  For 
example,  if  synergism  were  the  case,  subthreshold  doses  of  a  catecholamine  and  an 
endorphin  might  be  combined  in  the  same  micropipette  to  produce  effective 
reinforcement. 
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VH.  OBJECTIVES 


The  main  objectives  of  this  research  program  include  the  following; 
1)  demonstration  that  the  activity  of  single  neurons  can  be  reinforced  by  cellular 
applications  of  transmitters  or  drugs,  2)  determination  of  the  properties  and  limits 
of  such  neuronal  operant  conditioning,  and  3)  investigation  of  the  physiological  and 
biochemical  events  that  may  mediate  the  cellular  reinforcement  process.  These 
objectives  correspond  to  and  fall  naturally  into  three  phases  of  investigation.  As 
will  be  described  in  detail  below,  good  progress  has  been  made  in  accomplishing  the 
objectives  of  phase  l,  a  start  has  been  made  with  regard  to  the  objectives  of 
phase  2,  and  conceptual  work  has  begun  in  preparation  for  phase  3,  in  that  a 
plausible  and  testable  working  hypothesis  of  the  biochemistry  underlying  the 
cellular  reinforcement  process  has  been  formulated. 
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Vm.  METHODS 

1.  Surgery  and  Brain  Slice  Preparation 

In  whole  brain  experiments,  rats  were  anesthetized  with  chloral  hydrate, 
artificially  respirated,  and  placed  in  a  stereotaxic  instrument.  Using  a  Leitz 
micropositioner,  a  micropipette  was  lowered  into  different  brain  regions  through  a 
small  hole  in  the  skull,  and  slowly  advanced  until  suitable  action  potentials  were 
displayed  on  the  oscilloscope.  In  brain  slice  experiments,  rats  were  decapitated, 
and  their  brains  were  rapidly  removed  (30-45  sec)  and  chilled  to  6°  C  in  oxygenated 
artificial  cerebrospinal  fluid  (ACSF;  Dingledine,  et  al.,  1980).  Using  plastic  tools, 
the  hippocampal  region  was  dissected  out  and  rinsed  repeatedly  with  cold  ACSF  to 
minimize  cell  damage.  A  4-mm  block  was  isolated  by  parasagittal  cuts  using  a 
tissue  chopper.  One  cut  end  of  the  block  was  glued  with  cyanoacrylate  to  the 
bottom  of  the  cutting  chamber  and  supported  by  surrounding  it  with  agarose 
(1.8°/o,  43°  C).  Once  secured,  the  tissue  was  again  immersed  in  ACSF  at  6°  C  and 
further  sectioned  with  a  vibratome  to  yield  about  six  400-p  slices.  The  slices  were 
individually  transferred  to  a  static  chamber  and  supported  on  nylon  mesh  at  the 
surface  of  ACSF  solution  in  an  oxygenated  atmosphere  (95/5  O2/CO2,  500  ml/min) 
at  35°  C.  At  least  1  hr  of  incubation  was  allowed  for  recovery  of  physiological 
activity  prior  to  the  start  of  experiments  (Schwartzkroin,  1981;  Teyler,  1980). 
Fresh  ACSF  was  infised  into  the  static  chamber  every  30-45  min. 
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2.  Extracellular  Recording  and  Pressure  Microiniection 


Single-barrel  micropipette  blanks  (Omega  Dot)  were  pulled  on  a  Kopf 
electrode  puller  and  then  back-filled  with  test  solution  or  vehicle  (165  mM  saline). 
The  micropipette  was  connected  to  a  pressure  injector,  and  the  tip  broken  back 
under  microscopic  control  to  produce  a  droplet  18p  in  diameter  at  an  injector 
setting  of  15  p.s.i  and  35  ms.  Using  a  Leitz  micropositioner,  the  micropipette  was 
visually  guided  to  targeted  cells  and  slowly  lowered  until  a  suitable  action  potential 
was  obtained.  Unit  activity  was  displayed  on  a  Tektronix  5223  digital  storage 
oscilloscope,  monitored  on  a  loud  speaker,  and  recorded  on  a  Gould  220  strip  chart 
recorder.  Important  criteria  for  the  selection  of  suitable  cells  included  a 
signal/noise  ratio  of  at  least  4:1  and  relatively  stable  levels  of  baseline  activity. 
Action  potentials  were  led  into  a  Haer  74-45-1  amplitude  analyzer,  whose  output 
provided  digitized  input  to  the  computer  and  to  a  Haer  74-40-3  rate/interval 
analyzer.  Firing  rates  were  displayed  on  a  Houston  B-5000  strip  chart  recorder.  A 
Data  General  Eclipse  S/120  minicomputer  with  l28kB  of  memory  and  12.5MB  disk 
was  programmed  to  count  unit  activity  and  activate  the  injection  pump.  Both  the 
S/120  and  a  Data  General  Eclipse  S/140  computer  with  512kB  main  memory  and 
25MB  disk  was  used  for  on-line  data  storage  and  analysis.  An  Ampex  PR260 
7-channel  FM  recorder  provided  a  permanent  record  of  all  essential  experimental 
events  in  sequence  for  later  analysis. 


A  high-pressure  microinjection  system  (Picospritzer  II,  General  Valve  Corp.) 
was  used  for  rapid  extracellular  delivery  of  picoliter  volumes  of  neurotransmitters 
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and  drugs.  Pressure  injection  is  required  for  immediate  delivery  of  reinforcing 
solutions  with  injection  durations  as  short  as  5ms.  High  pressure  nylaflow  tubing 
was  used  to  connect  the  injection  pump  to  the  micropipette. 

3.  Single-Unit  Operant  Conditioning  Procedures 

A  somewhat  arbitrary  decision  was  made  in  choosing  which  aspect  of  unit 
activity  to  reinforce.  Since  firing  rates  are  likely  to  be  an  important  vehicle  for 
information  transmission,  peak  rates  should  have  high  information  value  and  might 
be  amenable  to  conditioning.  Thus,  in  these  experiments,  we  defined  a  half-second 
period  of  relatively  fast  activity  as  the  neuronal  response  to  reinforce.  These 
neuronal  responses  or  "bursts"  were  individually  determined  for  each  unit  studied. 
Prior  to  the  start  of  conditioning,  500  successive  half-second  samples  of  neuronal 
activity  were  recorded  and  a  frequency  distribution  of  the  number  of  spikes  per 
sample  was  compiled.  A  "burst"  was  defined  as  that  spike  number  that  was 
equalled  or  exceeded  in  only  2  percent  of  the  samples. 

In  a  few  experiments,  we  have  made  use  of  a  new  computer  program  to 
detect  bursts  of  firing  in  a  relatively  precise  way  (see  Appendix  II).  In  this 
program,  a  burst  is  defined  as  n  or  more  spikes  in  a  train  with  a  maximum 
interspike  interval  of  t  ms.  (In  the  case  of  the  CA1  pyramidal  neuron,  preliminary 
data  indicate  that  satisfactory  results  may  be  obtained  with  n  =  3  and  t  =  15ms, 
although  these  values  will  vary  somewhat  depending  on  the  neuron  under 
investigation).  Since  the  duration  of  each  burst  will  be  determined  precisely,  it 
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will  be  possible  to  reinforce  the  neuronal  response  immediately  or  at  a  fixed  delay 
after  its  completion.  (For  a  related  analysis  of  delay  of  reinforcement  at  the 
behavioral  level,  also  supported  by  this  project,  see  Black,  Belluzzi,  &  Stein, 
submitted  for  publication,  and  Appendix  I). 

The  basic  operant  conditioning  method  involved  six  stages:  1)  Baseline.  The 
number  of  "bursts"  in  the  absence  of  reinforcement  (operant  level)  was  determined 
during  a  baseline  period  of  approximately  10  minutes.  2)  Operant  Conditioning. 
Each  "burst"  was  now  followed  by  an  injection  of  the  reinforcing  solution.  If 
conditioning  failed  to  occur  after  5  minutes,  the  duration  of  the  injection  (and 
hence  the  dose)  was  increased  until  evidence  of  conditioning  was  obtained,  or  until 
direct  pharmacological  or  mechanical  effects  interfered  with  recording.  3) 
Extinction.  Reinforcement  was  terminated,  and  recording  continued  until  the 
baseline  was  recovered.  4)  Matched  "Free"  injections.  Noncontingent  injections  of 
the  reinforcing  solution  were  made  at  regular  intervals  to  determine  direct 
pharmacological  effects  on  rates  of  firing  and  probability  of  "bursts."  The  pattern 
and  number  of  "free"  injections  were  matched  to  the  pattern  and  number  of 
reinforcing  injections  in  the  preceding  phase  of  operant  conditioning.  The 
presentation  of  programmed  free  injections  was  delayed  for  three  sec  after  the 
occurence  of  "bursts"  to  minimize  their  adventitious  reinforcement.  5)  Washout. 
A  second  baseline  period  without  injections  was  given  to  allow  residual  effects  of 
the  noncontingent  drug  administrations  to  be  dissipated.  6)  Reacquisition. 
Whenever  possible,  a  second  period  of  reinforcement  was  scheduled  in  order  to 
compare  rates  of  original  acquisition  and  reacquisition. 
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IX.  RESULTS 

1.  Whole  Brain 

Initial  work  was  performed  in  the  whole  animal  to  maximize  the  probability 
of  successful  experiments.  In  the  absence  of  experience  with  locally  applied 
reinforcement,  it  was  felt  that  surgical  reduction  of  the  preparation,  or  the  use  of 
cells  in  culture,  should  be  deferred  until  positive  results  could  be  demonstrated  in 
the  intact  brain.  An  early  positive  experiment  in  an  intact,  anesthetized 
preparation  with  cocaine  (a  dopamine  enhancer)  as  the  reinforcing  solution  is 
displayed  in  Figure  2.  This  unit,  probably  located  in  dorsal  hypothalamus,  exhibited 
a  seven-fold  increase  in  the  number  of  "bursts"  and  a  two-fold  increase  in  overall 
firing  rate  following  approximately  20  cocaine  reinforcements.  The  same  number 
of  free  cocaine  injections  had  no  effect  on  the  number  of  "bursts"  or  overall  spike 
frequency.  Another  apparently  positive  experiment  with  amphetamine  (a  dopamine 
releaser)  as  the  reinforcing  agent  is  shown  for  a  unit  presumably  located  in 
hippocampus  (Fig.  3).  In  this  case,  a  dramatic  increase  in  the  rate  of  "bursts"  was 
observed  after  approximately  6  reinforcements.  Twenty  similar  amphetamine 
injections,  administered  noncontingently  in  the  free  injection  phase,  produced  no 
significant  increase  in  the  number  of  "bursts"  or  overall  spike  rate.  Introduction  of 
an  FR-2  schedule  caused  a  decline  in  "bursts"  in  this  and  other  experiments. 
Reintroduction  of  the  regular  reinforcement  schedule  increased  the  number  of 
"bursts"  which  again  decreased  when  the  FR-2  schedule  was  reinstated.  A  similar 
experiment  with  the  dopamine  receptor  agonist  apomorphine  serving  as  the 
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Figure  2 

Operant  conditioning  of  a  neuron  in  rat  brain  presumed  to  be  located  in 
dorsal  hypothalamus  using  local  injections  of  cocaine  as  reinforcement. 
Unit  activity  throughout  four  phases  of  a  complete  experiment  is 
shown.  Prior  to  the  first  baseline  phase,  the  "burst”  criterion  of  4  or 
more  spikes  per  half-second  sample  was  established.  Each  point  shows 
the  number  of  "bursts"  (lower  graph)  and  number  of  spikes  (upper  graph) 
in  blocks  of  100  half -second  samples  or  trials.  In  the  second  phase,  free 
injections  of  cocaine  HC1  (10  mM  in  165  mM  saline)  were  made  after 
every  20th  half-second  trial  for  a  total  of  20  free  injections.  The 
injection  duration  was  5  ms.  Following  a  second  baseline  period,  the 
5- ms  duration  cocaine  injections  were  delivered  after  each  "burst" 
(reinforcement  phase).  "Bursts"  and  overall  spike  rate  were  increased 
by  the  contingent  cocaine  injections  during  the  reinforcement  phase, 
but  were  not  increased  when  the  same  injections  were  administered 
noncontingentlv  in  the  free  injection  phase.  Insets  (upper  trace) 
photograph  of  digital  oscilloscope  display  of  two  action  potentials  from 
the  unit  undergoing  conditioning,  and  (lower  trace)  1-ms  time  markers. 
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Figure  3 

Operant  conditioning  of  a  presumed  hippocampal  neuron  in  rat  brain  using 
local  injections  of  amphetamine  as  reinforcement.  For  details,  see  text 
and  Fig.  2.  FR-1  =  regular  reinforcement  schedule,  FR-2  =  fixed-ratio 
2  reinforcement  schedule. 

reinforcing  drug  is  shown  for  a  unit  presumably  located  in  the  reticular  nucleus  of 
the  thalamus  (Fig.  4).  In  this  case,  an  increase  in  injection  duration  from  70  to  100 
ms  was  required  for  evidence  of  operant  conditioning.  Free  injections  of 
apomorphine,  both  preceding  and  following  the  reinforcement  phase,  failed  to 
increase  the  number  of  "bursts"  or  the  overall  spike  rate  above  baseline,  suggesting 
that  direct  stimulation  cannot  explain  apomorphine's  reinforcing  action.  Finally,  a 
negative  control  experiment,  in  which  saline  was  substituted  for  the  reinforcing 
drugs,  is  shown  in  Figure  5.  Mo  evidence  of  operant  conditioning  was  observed 
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Figure  4 

Operant  conditioning  of  a  presumed  thalamic  neuron  in  rat  brain  using 
local  injections  of  apomorphine  as  reinforcement.  For  details,  see  text 
and  Fig.  2.  At  D80  injection  duration  increased  from  70  to  80  ms,  at 
DtOO  injection  duration  increased  from  80  to  100  ms.  FR-2  =  fixed 
ratio  2  reinforcement  schedule,  FR-3  fixed  ratio  3  reinforcement 
schedule. 
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Figure  5 

Saline  control  experiment.  Failure  to  obtain  evidence  of  operant 
conditioning  of  a  presumed  cortical  neuron  in  rat  brain  using  local 
injections  of  saline  as  reinforcement.  For  details,  see  text  and  Fig.  2. 

suggesting  that  a  chemically  specific  message,  and  not  the  mere  contingency  of 
pressure  microinjection  following  bursts,  is  required  for  reinforcement. 

2.  Brain  Slices 

Contrary  to  initial  expectation,  our  most  satisfactory  experiments  have  been 
performed  in  brain  slices.  This  reduced  preparation  isolates  the  cell  undergoing 
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Figure  6 

Protocol  for  operant  conditioning  of  individual  neurons. 

conditioning  from  many  of  its  normal  connections.  Use  of  brain  slices  furthermore 
eliminates  the  artifacts  associated  with  anesthesia  and  the  possibility  that,  even 
under  anesthesia,  animals  may  be  influenced  by  environmental  stimulation.  Still 
another  major  advantage  of  the  brain  slice  technique  is  the  ability  to  return  to  the 
same  cell  type  from  experiment  to  experiment  since  microelectrode  placement  is 
under  direct  visual  control.  At  present  we  have  focused  our  efforts  on  hippocampal 
slices,  aiming  our  micropipettes  at  the  large  pyramidal  cells  in  the  CA1  field  of 
dorsal  hippocampus  (see  Fig.  6  for  a  diagram  of  the  hippocampal  slice  and 
experimental  protocol). 
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Figure  7 

Operant  conditioning  of  a  pyramidal  neuron  in  a  dorsal  hippocampal 
slice  using  local  injections  of  dopamine  as  reinforcement.  For  details, 
see  text  and  Fig.  2.  Free  =  free  injections  phase,  reinf.  =  reinforcement 
phase. 

A  positive  experiment  using  dopamine  as  the  reinforcing  solution  is  shown  for 
a  hippocampal  unit  in  Figure  7.  The  frequency  of  "bursts"  and  overall  firing  rates 
were  rapidly  increased  after  approximately  10  reinforcements  in  two  separate 
phases  of  operant  conditioning.  The  same  dopamine  injections  administered 
noncontingently  failed  to  increase  either  "burst"  frequency  or  overall  firing  rate. 
It  may  also  be  noted  that  extinction  occurred  rapidly  following  both  instances  of 
operant  conditioning.  A  second  positive  experiment  using  dopamine  is  shown  in 
Figure  8.  An  initial  period  of  free  injections  delivered  at  a  rate  of  approximately  5 
per  minute  had  no  effect  on  the  frequency  of  "bursts"  or  on  overall  firing  rate. 
During  a  first  phase  of  operant  conditioning,  "bursts"  and  firing  rates  were  sharply 
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INJECTIONS 

■LOCKS  OF  100  HALF-SECONO  TNIALS 

Figure  11 

Saline  control  exoeriment.  Failure  to  obtain  evidence  of  operant 
conditioning  of  a  pyramidal  neuron  in  dorsal  hippocampal  slice  using 
local  injections  of  saline  as  reinforcement.  For  details,  see  text  and 
Fig.  2. 

conditioning,  contingent  injections  of  dopamine  again  increased  the  frequency  of 
"bursts"  and  overall  firing  rate,  but  not  to  the  level  observed  in  the  first  phase  of 
reinforcement.  A  similar  pattern  of  results  was  obtained  in  an  experiment  with 
cocaine  (Fig.  9).  Injections  of  cocaine  during  2  phases  of  operant  conditioning 
sharply  increased  the  frequency  of  criterion  responses  and  the  overall  firing  rate, 
whereas  free  injections  had  no  facilitatory  effect  on  either  response  measure  and 
even  depressed  response  rates  below  the  baseline  level.  In  control  experiments, 
dopamine  was  administered  noncontingently  throughout  the  experiment  (Fig.  10)  or 
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Figure  12 

Summary  of  positive  dopamine  experiments, 
saline  was  substituted  for  dopamine  (Fig.  11).  In  neither  case  was  "burst" 
frequency  or  overall  firing  rate  increased.  We  have  also  observed  that  effective 
doses  of  reinforcing  solutions  in  hippocampal  slice  experiments  seem  to  be  lower  by 
an  order  of  magnitude  than  those  in  whole  brain  experiments. 


A  summary  of  8  positive  dopamine  experiments  is  shown  in  Figure  12. 
Plotted  here  are  the  means  of  the  peak  rates  obtained  in  each  phase  of  the 
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TREATMENT 
Figure  13 

Summary  of  positive  cocaine  experiments, 
experiment  for  each  neuron.  Highly  significant  increases  were  obtained  in  each  of 
the  reinforcement  periods  when  compared  either  to  control  periods  or  periods  when 
the  dopamine  was  presented  independently  of  neuronal  bursting.  A  similar 
summary  of  the  positive  cocaine  experiments  is  shown  in  Figure  13. 
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Finallv,  all  hippocampal  slice  experiments  containing  useful  data  are 
summarized  in  Table  2.  Excluded  from  this  Table  are  experiments  in  which 
suitable  action  potentials  could  not  be  obtained  or  held,  or  in  which  artifacts,  such 
as  drug  overdoses  or  clogging  of  the  micropipette,  caused  experiments  to  fail.  In 
the  columns  labelled  "RESULTS",  the  designations  are  as  follows:  +  = 

conditioning- like  changes  (increased  probability  of  bursts  following  reinforcement) 
plus  noncontingent  controls,  ?  =  conditioning-like  changes  but  no  controls,  and  - 
=  no  evidence  of  conditioning.  The  table  thus  indicates  that  9  of  the  17  dopamine 
experiments  were  positive  and  contained  noncontingent  controls.  According  to 
Benardo  <3c  Prince  (1982),  only  75  percent  of  CA1  hippocampal  cells  are  responsive 
to  dopamine  in  the  first  place;  hence,  our  success  rate  actually  represents  9/(17  x 
.75)  or  approximately  70  percent  of  all  completed  experiments.  We  consider  this 
rate  of  success  about  as  high  as  can  be  expected  at  this  stage,  given  the  difficulties 
involved.  In  the  cocaine  experiments,  the  success  rate  was  substantially  lower  than 
in  the  dopamine  experiments.  We  do  not  know  the  explanation,  but  speculate  that 
cocaine,  which  acts  via  dopamine,  may  require  a  physiologically  active  dopamine 
system  to  exert  its  full  reinforcing  effect.  Since  the  dopamine  axons  are  severed 
in  the  hippocampal  slice  preparation,  it  is  conceivable  that  their  responsivity  to 


cocaine  is  reduced. 
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X.  CONCLUSIONS 

After  a  long  phase  of  exploratory  experimentation,  we  have  found  that 
individual  neurons  mav  be  ooerantly  conditioned  by  direct  cellular  applications  of 
reinforcing  transmitters  or  drugs.  It  seems  unlikely  that  a  brain  cell  would  display 
a  gratuitous  capacity  for  operant  conditioning;  hence,  the  individual  neuron  mav 
represent  the  functional  unit  for  positive  reinforcement  in  the  brain. 

Applications  of  the  reinforcing  substances  had  opposite  effects  on  subsequent 
firing  rates,  depending  on  the  activity  pattern  of  the  neuron  at  the  time  of 
administration.  If  the  neuron  had  been  firing  rapidly  just  before  the  injection,  the 
firing  rate  was  increased;  if  the  neuron  had  been  firing  slowly  or  was  silent  just 
before  the  injection,  the  firing  rate  was  unaffected  or  decreased.  In  other  words, 
the  action  of  locally  applied  reinforcing  transmitters  or  drugs  on  brain  cells  was 
activity-related  in  a  way  that  formally  resembles  the  action  of  conventional 
reinforcers  on  behavior.  A  food  pellet  delivered  after  a  lever-press  response 
increases  lever  pressing,  whereas  the  same  pellet  delivered  independently  of  the 
behavior  has  no  effect  or  even  may  suppress  lever  pressing. 

We  have  begun  to  work  out  the  conditions  that  will  demonstrate  neuronal 
operant  conditioning  on  a  reliable  basis.  Thus,  we  find  at  present  the  most 
satisfactory  preparation  to  be  the  brain  slice,  the  best  neurons  for  operant 
conditioning  to  be  the  large  pyramidal  cells  in  the  CA1  field  of  dorsal  hippocampus, 
and  the  most  reliable  reinforcing  agents  to  be  dopamine  and  cocaine.  The 
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probability  of  CA1  firing  increased  sharply  when  reinforced  by  contingent 
applications  of  dopamine  applied  directly  to  the  cell  soma  by  pressure  injection; 
the  same  dopamine  injections  applied  independently  of  neuronal  firing  had  no  such 
effect  and,  in  fact,  tended  to  suppress  activity,  as  previously  reported  by  Benardo 
Sc  Prince  (1982).  The  reinforcing  action  of  dooamine,  which  is  a  long-term 
facilitatorv  effect,  must  thus  be  demonstrated  in  the  face  of  its  short-term 
inhibitory  action.  This  observation  raises  the  interesting  possibility  that  the  direct 
synaptic  action  of  reinforcement  transmitters  may  generally  be  inhibitory  rather 
than  excitatory. 

There  is  already  an  indication  of  dopaminergic  specificity:  included  among 
substances  that  were  ineffective  are  GABA,  serotonin,  acetylcholine,  imipramine, 
ethanol,  and  saline,  while  cocaine,  the  only  effective  reinforcer  other  than 
dopamine,  is  thought  to  act  by  enhancing  the  actions  of  endogenous  dopamine.  On 
a  preliminary  basis,  dopamine  seemed  to  be  a  more  effective  agent  than 
norepinephrine.  Some  evidence  suggests  that  reinforcement  schedules  may  have  a 
different  effect  in  single  neurons  than  they  do  in  the  whole  animal.  Even  a  simple 
fixed-ratio  2  schedule  caused  decrements,  rather  than  increments,  in  firing  rates. 
Our  neuronal  preparation  also  seemed  to  differ  from  conventional  operant 
conditioning  with  regard  to  reacquisition  after  a  period  of  extinction.  More  rapid 
reacquisition  or  "savings",  typically  observed  in  behavioral  experiments,  is  not 
observed  in  the  neuronal  studies.  The  effects  of  reinforcement  schedules  and 
reacquisition  thus  may  reflect  properties  of  neuronal  systems  rather  than  of 
individual  cells. 
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A  troublesome  feature  of  these  experiments  was  the  fact  that  relatively  high 
concentrations  (ImM)  of  dopamine  and  cocaine  were  required  for  reinforcement. 
However,  it  should  be  clear  that  total  drug  dose  is  determined  not  only  by  the 
concentration  of  the  solution  injected  but  also  by  other  injection  parameters,  such 
as  duration  and  volume.  Because  drug  injections  in  our  experiment  had  to  be 
delivered  to  individual  cells  in  close  contingency  to  bursts  of  activity,  it  was 
necessary  to  use  exceedingly  short  injection  durations  (5-20ms)  and  small  volumes 
(0.5-3  picoliters).  After  diffusion  to  action  sites,  these  minute  droplets  of  drug 
presumably  are  diluted  to  the  same  concentrations  effective  in  other  studies, 
where  lower  initial  concentrations  of  drug  are  applied  in  greater  volumes  and  for 
much  longer  durations. 

In  future  experiments,  we  plan  systematically  to  map  a  large  number  of  cell 
types  in  different  brain  areas  for  their  ability  to  undergo  operant  conditioning.  We 
also  plan  to  determine  the  chemical  specificity  of  single  unit  reinforcement  by 
investigating  a  wide  variety  of  transmitters  and  drugs.  In  a  later  stage  of 
investigation,  suitable  neurons  in  cell  culture,  which  are  not  organized  into 
physiologically  meaningful  networks,  will  be  tested  for  their  ability  to  undergo 
operant  conditioning  in  an  attempt  to  provide  final  proof  that  reinforcement  takes 
place  at  the  level  of  the  single  cell. 

We  have  begun  to  consider  the  biochemical  events  that  may  be  involved  in 
cellular  reinforcement.  What  is  required  is  a  mechanism  that  will  satisfy  the 
following  conditions:  l)  if  a  cell  fires  in  some  characteristic  pattern  of  activity, 
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such  as  a  "burst",  and  2)  if  the  cell's  (dopaminergic)  reinforcement  receptors  are 
then  activated,  then  and  only  then,  3)  will  the  receptor  or  pacemaker  proteins  that 
control  firing  be  modified  (in  all  probability,  by  phosphorylation)  so  that  the 
probability  of  a  "burst"  will  be  increased.  Clearly,  onlv  recently  active  cells  can  be 
eligible  for  reinforcement.  We  do  not  know  which  aspect  of  the  activity  pattern 
renders  a  cell  eligible  for  reinforcement.  Four  possible  markers  of  recent  activity 
are:  depolarization-induced  changes  in  membrane  associated  proteins,  K+  efflux, 
Na+  influx,  and  Ca++  influx.  As  a  working  hypothesis,  we  propose  that  the  influx  of 
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Ca++  with  each  "burst"  may  signal  eligibility  for  reinforcement  (Fig.  14).  Evidence 
(Krnjevic,  1983)  consistent  with  this  hypothesis  includes  the  following: 
1)  hippocampal  pyramidal  cells  exhibit  a  remarkable  Ca++  influx,  10-fold  greater 
than  any  other  cell  studied,  2)  this  high-rate  Ca++  influx  may  be  specifically 
associated  with  pyramidal  cell  bursting  activity  —  the  "burst"  must  contain  3  or 
more  spikes  to  be  effective.  It  has  been  shown  in  CA3  cells  that  the  first  two 
spikes  are  tetrodotoxin  sensitive,  and  presumably  mainly  mediated  by  sodium;  the 
third  and  subsequent  spikes  are  presumably  mainly  mediated  by  Ca++  (Wong  <5c 
Prince),  and  3)  activation  of  the  large  Ca++  current  by  stimulation  of  CA1  inputs 
has  an  abrupt  threshold,  both  for  stimulation  frequency  and  intensity;  such  abrupt 
thresholds  are  consistent  with  discrete  states  of  reinforcement  eligibility  and 
noneligibility.  We  further  assume  that  the  Ca++  binds  to  calmodulin  and  that  the 
Ca++-calmodulin  complex  interacts  with  dopamine-sensitive  adenylate  cyclase  in 
the  cell  body  so  that  the  cyclase  subsequently  produces  more  cAMP  in  response  to 
dopamine  (Greengard,  1978).  The  final  step  involving  phosphorylation  of  receptor 
or  pacemaker  proteins  may  be  jointly  catalyzed  by  both  cAMP -dependent  and 
calcium-ealmodulin-dependent  protein  kinases.  Precisely  such  joint  activation  of  a 
neuron-specific  protein,  Protein  I,  has  been  reported  by  Huttner,  DeGennaro  and 
Greengard  (1981).  A  similar  requirement  for  activation  of  receptor  or  pacemaker 
proteins  would  provide  the  necessary  specificity  for  reinforcement-induced  protein 
phosphorylation. 

In  summary,  by  positing  a  contingency  requirement  at  two  points  in  the 
biochemical  cascade,  the  proposed  mechanism  provides  two-fold  insurance  that 
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only  recently  active  cells  can  be  reinforced.  First,  because  Ca++  is  required  for 
activation  of  adenylate  cyclase,  it  insures  that  only  recently  active  cells  will 
generate  high  intracellular  levels  of  cAMP  in  response  to  dopamine.  And  secondly, 
by  requiring  the  joint  action  of  calcium-calmodulin-dependent  and 
c AMP-dependent  protein  kinases  for  reinforcement-induced  modification  of  the 
receptor  and  pacemaker  proteins  that  control  firing,  it  again  insures  that  Ca++ 
influx  and  dopamine  receptor  stimulation  must  be  contingent. 

We  have  formulated  a  working  hypothesis  of  the  reinforcement  mechanism 
sufficiently  detailed  to  permit  experimental  testing.  Needless  to  say,  this 
hypothesis  will  be  modified  by  experimental  results.  Major  experiments  to  be 
performed  include:  1)  substitution  of  cAMP  for  dopamine  in  neuronal  operant 
conditioning  studies  to  determine  if,  in  fact,  dopamine's  action  is  mediated  via 
cAMP,  2)  Drevention  of  neuronal  operant  conditioning  by  use  of  calcium-free 
media,  or  by  pretreatment  with  calcium  chelators  or  calcium  channel  antagonists 
such  as  nitrendioine  to  determine  if,  in  fact,  Ca++  influx  is  required  for 
reinforcement,  3)  blockade  of  neuronal  operant  conditioning  by  the  highly  specific 
calmodulin  inhibitor  R-24571  (Van  Belle,  1981)  to  determine  if.  in  fact, 
reinforcement  is  a  calmodulin-dependent  process,  4)  substitution  of  a  calcium 
channel  activator  IBAY  K8644)  for  spontaneous  firing  in  neuronal  operant 
conditioning  experiments.  If  silent  cells  could  be  treated  with  BAY  K8644  to 
mimic  a  burst-induced  influx  of  Ca++,  and  if  these  cells  could  then  be  reinforced 
with  dopamine  (that  is,  if  they  later  exhibited  enhanced  bursting),  this  would  be  a 
powerful  oroof  that  the  influx  of  Ca++  is  the  biochemical  marker  for  reinforcement 


Stein,  L.  and  Belluzzi,  J.D. 


Final  Technical  Report  F49620-81-K-0015 


eligibility.  Finally,  in  vitro  phosphorylation  studies  of  hippocampal  tissue  may 
permit  us  to  identify  those  proteins  which  are  specifically  phosphorylated  by 
combined  administration  of  calcium-calmodulin  and  cAMP.  Such  experiments 
could  help  us  to  identify  the  reinforcement-sensitive  receptor  and  pacemaker 
proteins  that  control  cellular  firing  rates.  Furthermore,  if  these  proteins  could  be 
purified,  antibodies  could  be  raised  to  reveal  their  anatomical  localization  in  the 
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Xin.  APPENDIX  I:  Ancillary  Experiments 

Delay  of  Reinforcement 

The  effect  of"  delayed  reinforcement  on  the  acquisition  of  lateral 
hypothalamic  self-stimulation  was  investigated.  Brain  stimulation  reinforcement 
minimizes  cues  associated  with  reinforcement  delivery  (secondary  reinforcement) 
and,  by  eliminating  consummatory  responses,  permits  precise  temporal  control  of 
the  interval  between  the  operant  response  and  reinforcement.  Different  groups 
were  trained  in  daily  1-hr  sessions  for  brain  stimulation  reinforcement  at  one  of 
four  delay  intervals  (1,  2,  3,  or  6  sec).  Responses  made  during  the  delay  interval 
were  not  reinforced  and  reset  the  delay  timer.  Control  groups  were  reinforced 
immediately,  but  were  required  to  space  responses— according  to  a  DRL  schedule— 
for  an  interval  corresponding  to  one  of  the  delay  of  reinforcement  intervals.  The 
DRL  schedule  equalized  opportunities  for  reinforcement  and  non-reinforcement. 
At  all  intervals,  rats  trained  with  delayed  reinforcement  had  significantly  lower 
bar-press  rates  than  controls  trained  under  DRL.  When  reinforcement  schedules 
were  switched  (DELAY  groups  now  get  DRL  and  vice  versa),  response  rates  rapidly 
shifted  to  levels  appropriate  to  the  new  schedule.  The  pre-switch  results  indicate 
that  delays  even  as  short  as  1  second  markedly  impede  the  acquisition  of  self¬ 
stimulation  behavior.  The  post-switch  results  suggest  that  delay  of  reinforcement, 
like  stimulation  intensity,  may  determine  the  strength  of  hypthalamic 
reinforcement  and  hence  final  levels  of  performance. 
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Cell  Culture  Experiments 

A  number  of  experiments  were  carried  out  to  develop  a  neuronal  cell  culture 
model  in  which  operant  conditioning  could  be  tested.  The  requirements  to  be  met 
by  this  system  were:  (i)  cells  should  exhibit  spontaneous  electrical  activity,  (ii)  they 
should  possess  opiate  and  catecholamine  receptors,  (iii)  cells  should  be  large  enough 
to  permit  lengthy  recording  times  with  an  intracellular  electrode. 

Our  first  approach  was  to  use  the  neuroblastoma  x  glioma  hybrid  cell  line 
NG108-15,  a  tumor  derived  cell  line,  the  properties  of  which  satisfy  conditions  (ii) 
and  (iii).  Our  goal  was  to  manipulate  the  chemical  environment  of  the  NG108-15 
cells  in  order  to  induce  spontaneous  electrical  activity,  since  cyclic  AMP  (cAMP)  is 
thought  to  be  involved  in  the  process  of  neuronal  cell  maturation  and  the 
expression  of  electrical  activity.  Firstly,  cells  were  grown  in  a  medium  containing 
dibutyryl  cAMP  -  an  analogue  of  cAMP.  Secondly,  cells  were  cultured  in  the 
presence  of  prostaglandin  E  and  Theophylline,  which  increase  endogenous  cAMP 
levels.  NG108-15  cells  were  then  tested  electrophysiologically  for  spontaneous 
activity;  none  was  detected,  however. 

Another,  alternative  approach  was  attempted.  Since  single  cell  operant 
conditioning  was  detected  in  the  cerebellum  of  adult  rats,  we  embarked  on  a  series 
of  experiments  to  develop  a  cell  culture  preparation  of  hippocampal  and  cerebellar 
cells.  Hippocampus  and  cerebellum  of  neonatal  animals  were  dissected  out  into  a 
cell  suspension  and  plated  in  standard  media  for  primary  neuronal  cultures.  The 
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use  of  neonatal  material  was  essential  since  neurons  from  adult  animals  do  not 
survive  the  dissociation  procedure.  Our  aim  was  to  find  conditions  which  would 
enable  neurons  to  survive  in  cell  culture  for  prolonged  periods  of  time  in  order  to 
differentiate  and  acquire  electrical  activity  characteristic  of  neurons  in  adult 
animals.  Difficulties  were  encountered,  such  as  glial  cell  proliferation,  which 
tended  to  decrease  neuronal  survival.  This  was  overcome  by  use  of  mitotic 
inhibitors.  Neuronal  cultures  from  hippocampi  and  cerebella  grown  in  culture  for 
up  to  4  weeks  were  finally  obtained  and  these  were  tested  electrophysiologically. 
These  cells,  however,  proved  to  be  too  small  and  fragile  to  survive  microelectrode 
impalement  and  attempts  to  obtain  recordings  from  them  proved  unsuccessful. 

We  intend  to  continue  our  search  for  a  viable  cell  culture  model  by  using 
approaches  such  as  cell  hybridization.  This  will  involve  fusing  two  different  cell 
types  each  of  which  has  some,  but  not  all,  of  the  desirable  properties  outlined 
above.  A  hybrid  will  be  created  which  will  hopefully  retain  all  the  properties  of  its 
parent  cell  lines. 


Studies  on  Calcium  Channels. 

A  crucial  role  for  calcium  as  the  signal  for  reinforcement  eligibility  has 
already  been  discussed.  Calcium  channels  in  the  periphery  can  be  inactivated  by 
the  calcium  channel  blockers  D600  (methoxyverapamil)  and  nitrendipine  (DHP). 
However,  D600  is  not  suitable  for  neuronal  operant  conditioning  studies  because 
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the  drug  also  affects  radioligand  binding  of  brain  muscarinic,  ax-adrenergic  and 
opiate  receptors.  It  was  thus  of  interest  to  determine  whether  DHP  drugs  have 
more  selective  calcium-channel  blocking  actions  without  actions  on  brain 
receptors. 

The  DHP  drugs  used  in  this  study  were  nicardipine,  nitrendipine,  nimodipine, 
felodipine,  nifedipine  and  nisoldipine.  They  were  examined  for  activity  in 
inhibiting  the  specific  binding  of  ^H-QNB  and  ^H-WB40t  to  the  muscarinic  and  a^- 
adrenergic  receptors  of  rat-brain.  It  was  found  that  nicardipine  was  the  most 
active  of  these  DHP  drugs  in  inhibiting  receptor  binding  function  and  that  this 
inhibition  was  competitive  and  stereoselective,  with  the  (+)  isomer  of  nicardipine 
being  28  times  as  active  as  the  (-)  isomer  on  the  brain  muscarinic  receptor  and  3 
times  as  active  on  the  ax-receptor.  This  stereoselectivity  suggests  direct 
interaction  with  the  receptors.  An  allosteric  effect  occurring  indirectly  at  the 
receptors  via  a  primary  reaction  with  the  DHP  high-affinity  binding  site 
presumably  associated  with  a  brain  Ca++  channel  was  ruled  out  experimentally. 
These  results  suggest  (a)  that  nicardipine  should  not  be  used  in  brain  reinforcement 
studies  where  effects  due  solely  on  Ca++  channels  are  being  sought,  and  (b)  other 
DHP  drugs  such  as  nifedifine  or  nisoldipine  would  be  better  choices  in  such 
experiments,  since  they  have  appreciably  less  effect  on  brain  receptors 
(Biochemical  Pharmacology,  submitted). 
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1.0  INTRODUCTION 

1 . 1  SYSTEM  OVERVIEW 

The  NCCP  System  is  designed  to  provide  the  capability  to  perform 
real-time  data  acquisition,  analysis,  control,  and  reporting  of 
Neuronal  Response  Tests  and  non  real-time  data  reduction,  program 
development,  and  computation.  The  system  is  based  on  Data 
General's  S/120  Eclipse  Computer,  Data  General's  ROOS  operating 
system,  and  XYCOM’s  Neuronal  Conditioning  Control  software. 

Digital  inputs  and  outputs  are  made  through  a  Data  General  Digital 
I/O  Card  (4040).  Digital  inputs  can  be  pulses  or  switches,  and  the 
outputs  can  be  used  for  controlling  or  monitoring  the  experiment. 

1.2  SCOPE  OF  DOCUMENT 

This  manual  is  limited  in  scope  to  describing  and  explaining  the 
set-up  and  operation  of  the  NCCP  software.  Detailed  descriptions  of 
the  computer  hardware  and  peripherals  as  well  as  operating  RDOS  are 
contained  in  the  manuals  provided  by  Data  General. 


2.0  HARDWARE  DESCRIPTION 
2.1  HARDWARE  CONFIGURATION 

The  NCCP  system  is  based  on  Data  General  Hardware,  Data  General 
operating  system  software,  and  XYCOM  application  software.  Figure 
1  is  a  block  diagram  of  the  hardware  and  is  comprised  of  the 
following  components: 
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The  experiment  interfaces  to  the  NCCP  through  the  digital  I/O  lines 
which  require  standard  TTL  logic  levels,  and  connect  to  the  ACT 
boards  to  provide  -24VDC  inputs  and  outputs  to  the  environment. 
Input/Output  lines  can  be  assigned  anywhere  provided  the  software 
definitions  are  changed  accordingly.  Current  assignments  are  shown  below. 
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2.2  LINE  ASSIGNMENTS: 


ACT 

NO. 

OCTAL 

VALUE 

BIT 

INPUT  LINE 
FUNCTION 

200 

100000 

00 

_ 

100 

040000 

01 

— 

40 

020000 

02 

— 

20 

010000 

03 

— 

10 

004000 

04 

— 

4 

002000 

05 

Manual  Switch 

2 

001000 

06 

“B”  spikes 

1 

000400 

07 

"A"  spikes 

OUTPUT  LINE 
FUNCTION 


Spike  Out  (1msec  echo) 

Reinforcement  Duration  Indicator 

Reinforcement  Pump  On 

Print  Marker 

Spike  in  W i ndow 

Wi ndow  Over 

Burst  Found 

-Spare- 


3.0  SYSTEM  OPERATION 
3.1  Startup 

1.  Turn  Cabinet  Power  ON 

2.  Turn  Computer  Power  ON 

3.  Turn  Oisk  Power  ON 

4.  Turn  Printer  Power  ON 
Adjust  Printer  Top  of  Form 
Press  Printer  "On-Line" 

5.  Turn  Terminal  Power  ON 
It  should  respond: 

"D2 10  Self  Test  OK" 

Press  "Alpha  Lock"  key 

6.  After  the  disk  unit  ready  light  comes  "ON" 

Press  Computer  "Reset" 

Press  Computer  "PR LOAD" 

7.  The  Terminal  should  now  be  displaying 

F I LENAME 

Type  in  a  "NEW  LINE* 

(From  now  on,  "NEW  LINE"  will  be  shown  as  (ND) 

8.  The  system  will  request  date  &  timet  respond  accordingly. 
Terminate  each  complete  entry  with  a  (NL) 

9.  The  system  will  now  display  an  ”R"  for  "READY" 

Type:  UP(NL) 

The  "UP"  command  clears  discrete  outputs,  clears  files, 
initializes  the  system  and  the  UCI  directories,  and  puts  the 
system  in  the  UCI  directory.  At  this  point,  any  ROOS  command 
or  program  can  be  executed  or  the  Neuronal  Conditioning  Control 
Program  can  be  executed  as  described  in  Secton  3.2. 


3.2  Test  Operat ion 

The  program  is  initiated  by  typing 
NCCP(NL) 

This  causes  the  mam  program  to  be  read  into  memory,  common  area  to 
be  cleared,  and  default  values  to  be  initialized.  The  setup  overlay 
(PRT3T1)  is  then  loaded  and  executed.  Setup  displays  the  menu 
shown  in  Figure  2.  A  description  of  each  selection  is  on  the  next  page. 
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FIGURE  2:  MAIN  MENU 


NCCP  PRERUN  MENU 


EDIT  SETUP  -  1 
PRINT  SETUP  -  2 
RUN  TEST  -  3 
HELP  (DRUGS)  -  4 
HELP  (COMMANDS)  -  5 
EXIT  -  6 


SELECTION  -  _ 

(ERROR  IN  SELECTION  -  SELECT  1  -  6) 

3.3  MAIN  MENU  DESCRIPTION 

1  -  EDIT  SETUP  displays  the  current  value  of  all  parameters  that  can 

be  modified  by  the  operator.  Figure  3  shows  the  screen  after 
this  selection  has  been  made.  The  cursor  is  then  positiond  at 
the  first  entry  and  the  operator  can  effect  changes  to  the  setup 
by  keying  in  the  new  value  followed  by  a  New  Line.  The 
cursor  will  then  advance  to  the  next  parameter. 

If  no  change  is  required,  just  key  in  a  (NL)  without  a  preceding 
value.  Keying  in  an  (ESC)  will  cause  the  EDIT  program  to 
return  to  the  menu.  All  changes  entered  prior  to  the  (ESC)  will 
be  in  effect. 

2  -  PRINT  SETUP  prints  the  setup  information  on  the  printer  in  the 

format  shown  in  Figure  3. 

3  -  RUN  TEST  causes  the  SETUP  program  to  be  exited  and  actual 

testing  to  begin  using  the  parameters  defined  by  SETUP.  Each 
time  you  begin  testing,  the  computer  will  request  the  name  of 
the  file  you  wish  to  use  to  store  your  test  data.  Up  to  20 
characters  are  allowed.  Once  a  file  has  been  used,  it  cannot  be 
reused  until  you  delete  it  wih  the  ROOS  "DELETE"  command. 
STOPPING  the  test  is  accomplished  through  an  on-line  CLI 
command. 

4  -  HELP  (DRUGS)  displays  all  of  the  drugs  defined  to  the  system  and 

their  corresponding  symbols  to  be  used  during  setup.  To 
continue  after  viewing  the  display,  strike  any  key  (see  Figure 

4)  . 

5  -  HELP  (COMMANOS)  displays  all  of  the  on-line  commands  available 

to  the  operator  and  a  brief  description  of  each  command.  To 
continue  after  viewing  the  display,  strike  any  key  (see  Figure 

5) . 

6  -  EXIT  causes  the  NCCP  program  to  stop  and  control  is  returned  to 

ROOS. 

Each  command,  except  3  &  6,  returns  to  the  menu. 
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FIGURE  3:  EOIT  SETUP  PRERUN  SETUP 


RAT  *  UNIT  • 

DRUG  10 

OOSE  PS  I 

WINDOW  DESCRIPTION 

MODE  (M/F):  M  SIZE:  500  WINDOWS/PRINT:  100  INTERSPIKE  INTERVL :  10 
HISTOGRAM  DESCRIPTION 

BIN  SIZE:  300  *  OF  BINS:  10  DURATION  (D3)  3000 

REINFORCEMENT  DESCRIPTION 

MOOE  (R.B.F, )  B  WINDOW/FREE  REINF.  40  CRITERIA:  3 

RATIO:  1  INJ  DURATION:  50 

DELAYS 

POST  WINOOW  (01)  0  POST  REINFORCEMENT  (D2)  0 

(CR)  ->  NO  CHANGE.  (ESC)  ->  RETURN  TO  MENU 

FIGURE  4:  DRUG  COOES 


CODE 

DRUG 

CODE 

DRUG 

CODE 

DRUG 

01 

Acetyl cho 1 

08 

Enkephal in 

15 

Norep i neph 

02 

Amphetami ne 

09 

Endorph i n 

16 

Sal  i  ne 

03 

Apomorphine 

10 

GABA 

17 

Serotonin 

04 

Car bachol 

1 1 

Glut  ama  t  e 

18 

Tof  ran i  1 

05 

Ch 1  or promaz 

12 

Imiprami ne 

19 

Other 

06 

Coca i ne 

13 

Methamphet 

20 

EOD 

07 

Dopami ne 

14 

Morph i ne 

FIGURE  5:  COMMAND  TABLE 


CODE 

FUNCTION 

RANGE 

OEFAULT 

BM 

Burst  Submode 

(F 

or  M) 

F 

CR 

Set  Criterion  (•  spikes  for  rein) 

( 1 

-  100) 

3 

D 1 

Set  Post  Window  Delay 

(0 

-  500) 

0 

02 

Set  Post  Reinforcement  Delay 

(0 

-  500) 

0 

D3 

Hi stogram  Durat ion 

( 1 

-  10000) 

3000 

FW 

Set  Fixed  Window  Mode  (msec) 

( 10 

-  2000) 

500 

HB 

Set  Number  of  Histogram  Bins 

(1 

-  50) 

10 

HE 

Display  Command  HELPS 

— 

— 

HS 

Set  Histogram  Bin  Size  (msec) 

(1 

-  200) 

100 

1  1 

Interspike  Interval  (msec) 

(5 

-  100) 

10 

MD 

Set  MOOE  (Treatment) 

(B. 

R,  F) 

B 

MW 

Start  Moving  Window  "BURST"  Mode 

( 10 

-  2000) 

500 

RC 

Graphics  Dump  to  Printer 

— 

— 

RD 

Set  Reinforcement  Duration  (msec) 

(5 

-  1000) 

50 

RP 

Replot  Graph  on  the  Terminal 

— 

— 

RT 

Set  Ratio  (Bur st s/ re i nf ) 

( 1 

-  99) 

1 

ST 

Stop  Test 

— 

— 

WC 

Set  Windows/Free  Reinforcement 

( 1 

-  100) 

40 

WP 

Set  Wi ndows/Pr i nt 

(10 

-  200) 

100 

WS 

Set  Window  Size  (msec) 

— 

-- 

XX 

Exit  CLI  Active  Mode 

(0 

-  500) 
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3.4  COMMAND  LINE  INTERPRETER  (CL  1 3 

Following  initiation  of  a  test,  the  screen  is  used  for  two  functions: 

1)  to  display  real  time  events  as  described  later,  and  2)  to  display 
operator  commands.  In  order  to  prevent  the  two  functions  from 
interfering  with  each  other,  two  separate  tasks  have  been  created 
and  a  control  algorithm  has  been  implemented  that  allows  only  one  or 
the  other  to  be  in  control  of  the  screen. 

COMMAND  ENTRY: 

The  CLI  task  is  initiated  by  the  main  program  which  in  turn  issues  a 
request  for  an  input  from  the  keyboard.  To  make  a  request  for  a  command 
entry  the  operator  must  key 

<CNTL  W>  or  the  up  arrow. 

Any  other  entry  is  ignored  (except  CLI  commands,  e.g.,  CNTL  A,  etc.) 

After  finding  a  CNTL  W,  CLI  will  check  the  HRT  Display  Request"  flag 
which  will  be  false  from  the  time  the  real  time  display  completes 
its  update  until  10  seconds  before  the  next  update.  If  it  is  false, 

CLI  will  display  "=>"  on  line  22,  sound  the  terminal  bell  (beep) 
and  set  the  "terminal  in  use"  flag.  At  this  time,  the  operator  can 
enter  commands.  Every  IS  seconds  CLI  will  check  to  see  if  a 
command  has  been  input  and  if  not,  it  will  release  the  display  and 
the  operator  must  key  in  another  <CNTL  W> .  If  the  CLI  input  task 
was  terminated  because  of  the  timeout  feature,  then  the  CNTL  W  must 
be  preceded  by  a  character  (any  alpha/numeric)  before  the  CNTL  W 
will  be  recognized  by  CLI. 

3.4.1  IMMEDIATE  COMMANDS: 

Normally,  changes  in  SETUP  parameters  do  not  take  effect  until  the 
next  print  marker  occurs;  however,  provision  has  been  made  to  allow 
instantaneous  action.  For  numeric  values,  simply  key  in  a  negative 
value  and  for  non-numeric  values,  follow  the  letter  with  an  I. 

Figure  5  lists  all  of  the  on-line  commands  and  a  one  sentence 
description  and  the  following  paragraphs  expand  the  description  when 
appl icable. 

BM  -  The  BURST  MODE  of  detecting  cell  activity  has  two  submodes,  ALL 
(A)  and  MINIMUM  (M).  Every  test  starts  in  the  "ALL"  mode  and 
the  mode  can  be  changed  by  the  BM  command.  MINIMUM  means  that 
a  Burst  will  be  detected  when  the  number  of  consecutive  pulses 
that  are  separated  by  less  than  the  "interspike  interval"  has 
reached  the  criteria  level.  ALL  means  that  a  Burst  wi 1 1  be 
detected  when  the  number  of  consecutive  pulses  that  are 
separated  by  less  than  the  "interspike  interval"  has  reached 
the  criteria  level  and  that  "interspike  interval"  milliseconds 
has  elapsed  since  the  last  pulse  has  occurred. 
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CR  -  SET  CRITERION 

CR  specifies  the  number  of  pulses  (spikes)  required  within  a 
window  before  the  system  will  count  a  burst  and  apply 
reinforcement.  Other  parameters,  such  as  mode  or  ratio,  could 
prevent  reinforcment  from  occurring. 

01  -  Set  POST-WINOOW  DELAY 

This  Oelay  is  applied  after  a  window  has  been  detected  and 
before  reinforcement  is  started. 

02  -  Set  POST-REINFORCEMENT  DELAY 

This  Oelay  is  applied  after  reinforcement  is  turned  off  and 
before  starting  Histogram  data  collection. 

03  -  HISTOGRAM  DURATION  OELAY 

This  Delay  is  applied  after  beginning  Histogram  collection  and 
is  used  by  the  system  to  determine  when  to  start  searching  for 
another  window.  03  must  be  greater  than  or  equal  to  Histogram 
bin  size  (HS)  times  the  number  of  Histogram  bins  (HB). 

FW  -  FIXED  WINDOW 

Starts  "FIXED"  window  and  allows  window  duration  (in  msec)  to  be 
changed.  A  "Burst"  equals  criterion  number  (CR)  of  spikes  falling 
in  the  window,  irrespective  of  interspike  interval  (II). 

HB  -  Set  the  NUMBER  OF  HISTOGRAM  BINS 

A  "Bin"  is  a  period  of  time  equal  to  the  value  defined  below 
under  HS.  Bins  occur  during  D3. 

HE  -  Display  the  "HELP  Commands"  display. 

HS  -  Set  the  duration  (in  msec)  of  each  Histogram  bin. 

II  -  Set  the  Interspike  Interval  (in  msec). 

MD  -  Change  MODE  to  BASELINE  (B),  REINFORCE  ON  (R).  OR  FREE 
INJECTIONS  ON  (F) . 

MW  -  MOVING  WINDOW 

Sets  the  window  MODE  to  "BURST"  and  allows  the  window  duration 
(in  msec)  to  be  changed  for  print  determination.  There  will  be 
a  printout  every  N  windows,  where  N  is  set  using  the  WP  command. 

RC  -  PRINT  GRAPHIC  DISPLAY  on  the  printer.  If  this  is  done 
while  windows  are  still  being  located  and  printed,  this 
printout  may  by  interspersed  with  the  window  data. 

R0  -  Set  the  REINFORCEMENT  OURATION  (in  msec). 

RP  -  REDISPLAY  GRAPH  PLOT  on  the  CRT  terminal  from  the  beginning. 

RT  -  Set  the  REINFORCEMENT  RATIO,  i.e.,  i  Bursts/reinf . 

ST  -  STOP  TEST t  requests  post-test  comments,  and  then  returns 
to  the  menu. 
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WC  -  Set  WINDOWS/FREE  REINFORCEMENT,  i.e.,  the  number  of  windows 
between  consecutive  free  injections. 

WP  -  Set  WINDOWS /PR  I  NT, i .e. ,  the  time  between  printouts. 

WS  -  Set  WINOOW  DURATION  (in  msec). 

XX  -  EXIT  COMMAND  entry  mode  and  allow  the  graphics  display  to  return. 
4.0  OPERATIONAL  SOFTWARE 

This  section  is  designed  to  present  the  users  with  an  overview  of 
the  NCCP  software  in  order  to  assist  the  user  in  making  simple 
changes.  A  complete  list  of  all  the  NCCP  programs  is  shown  in 
Appendix  I,  and  each  module's  source  file  (.FR)  contains  header 
information  that  lists  or  describes  the  functions  of  the  module.  In 
addition,  most  modules  contain  comments  to  describe  the  FORTRAN 
commands  to  facilitate  understanding  of  the  program. 

The  NCCP  software  runs  under  ROOS  and  is,  therefore,  subject  to  all 
of  the  restrictions  and  limitations  of  RDOS.  Figure  6.1  is  a  block 
diagram  representative  of  the  software  structure.  NCCP  operates  in 
two  basic  modes,  SETUP  and  TBST  (see  Flowchart,  Figure  6.2).  In 
the  SETUP  mode,  pulse  detection  is  not  functional  and  the  operator 
can  define  the  test  parameters.  After  SETUP,  NCCP  goes  into  the  TBST 
mode,  and  three  major  functions  are  then  activated: 

RTCON  -  Test  Control 

RTCLI  -  Real  Time  Command  Line  Interpreter 

REAL  -  Real  Time  Processing 

RTCON  performs  initialization  of  parameters  and  starts  the  real  time 
process.  It  then  waits  for  a  response  from  the  real  time  process 
and  performs  the'post-event  processing.  Events  are  windows,  bursts, 
and  histogram  completions  (see  Flowchart,  Figure  6.2). 

REAL  is  an  assembly  language  program  that  is  connected  to  the  RDOS 
clock  interrupt  routiner  therefore,  it  runs  at  the  interrupt  level 
and  at  the  1  KHZ  rate.  REAL  handles  all  input  and  output  discrete 
processing,  detection  of  BURSTS  (if  in  BURST  mode),  timing  of 
durations,  counting  pulses,  and  communicating  to  RTCON  the  status  of 
real-time  activity. 

RTCLI  allows  the  operator  to  interact  with  the  test  as  the  test 
progresses.  Most  parameters  and  modes  can  be  changed  through 
individual  commands  as  described  previously. 

As  the  test  runs,  data  is  saved  in  a  data  file  whose  name  is: 

RAT  NO.  I  UNIT  .0T  (e.g.,  123461. DT) 

The  structure  of  the  data  file  and  tts  records  is  shown  in  Appendix 
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4.1  SOFTWARE  BLOCK  DIAGRAM 

ir 


SETUP 


RDOS 


NCCP 


RTCLI 


I -Change 

Parameters 


i  REAL-TIME 
> !  CLOCK 


-Burst  Detection 
-Window  Timing 
-Re i nf or  cement 
-Delay  Timing 
-Histogram 
-Pulse  counting 


RTCON 


-Wi ndow  Process i ng 
Summary  Print 
Wi ndow  Print 
Plot 

-Burst  Processing 
-Histogram  Processing 
-Status  Print 


FIGURE  6.1 

SOFTWARE  BLOCK  DIAGRAM 
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4.2  MAIN  PROGRAM  (NCCP)  FLOWCHART 


(3)- 


(2) 

i 

i 

V 

KILL 

TASKS 


/  EOTPRO  \ 

/ _ \ 

/  ENO  OF  TEST  \ 
\  PROCESSING  / 

\ - V - / 

(3) 


SETUP, ETC.  / 
_ / 


/  PRTST2  \ 

/ . \ 

/  CREATE  &  \ 

\  OPEN  DATA  / 
\  FILE  / 

\ _ / 

.YES. 


LOAD  PLOT 
ROUTINES 
INTO  OVERLAY 
AREA 


SET  TEST  ON 
SCHEDULE  RTCON 
START  CLOCK 
-V. 

(1) 


:  nccp  : 

i  __ 

CD 

V  ... 

_V_ 

/  DIOOF 

/  \ 

/ - 

/  \ 

/  DEFINE 

\ 

<TEST  ON 

\  DISCRETES 

/ 

\  / 

\  DEVICE 

/ 

\ _ / 

V  / 

YES  : 

1 

i 

_v_ 

V 

/  \ 

/  DFAULT 

\ 

/  \ 

/ - 

<  WRITE 

/  SET  UP  NCCP 

\ 

\  DATA/ 

\  DEFAULT 

/ 

\ _ / 

\  VALUES 

/ 

YES  : 

\ 

/ 

V 

- > ; 

:  WRITE 

V 

:  DATA  TO 

/  PRTST1 

\ 

:  DISK 

/ _ _ 

-\ 

1 

/  MAIN  MENU 

!  <  — 

NO 


_V_ 

/  \ 

NO  /  \ 

(--.-<SCREEN  > 

\  FREE/ 

\ _ / 

YES  ! 

_ V _ 


SET 

SCREEN 
"IN  USE" 


CPLOT 


/  PLOT  CRT  \ 
\  IF  REQUIRED  / 

\ _ / 


RESET 
SCREEN 
"IN  USE" 


FIGURE  6.2 

MAIN  PROGRAM:  NCCP 
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4.3  CONTROL  PROGRAM  (RTCON)  FLOWCHART 
.*  RTCON  !  (2 


/  INITRC  \ 

/ - \ 

/  INITIALIZE  \ 

\  TEST  / 

\  PARAMETERS  / 

\ _ / 

:  < . m 

_  .  _v_ 

/  \  /  \ 

YES/  STS  \  NO  /PARA-\ 

—  <PRINTING<  — <  METER> 

!  \  DONE/  \ CHANGE 

I  \ _ /  \ _ / 

:  !  NO  :  YES 

:  :  v _ 

:  :  MOVE  PRESENT  *. 

:  i  !  BUFFER  TO 

:  :  :  setbuf  &  : 

;  :  : setup  parameters: 

:  :  :  for  clock  rout.: 


- > : 

_v_ 

/  \ 

)  /  \ 
—  <  WINOOW) 
\COUNT / 

\  =  0  / 
\_/ 

:  YES 
V 

/  NCRSTS 


\  PARAMETER 
\  STATUS 


!  IF  MW  PULSR 
!  IF  FW  PULSR 


(2) 

:  BURST 

_ V _ 

UPDATE  ! 

TOTAL  1 
SPIKES  : 


WINDOW 


SET  "WINDOW 
OVER"  PULSE 
INCREMENT  FREE 
REINFORCE  & 
PRINT  WINDOW 
COUNT 


_V_ 

/  \ 

/MOVING  YES 

<  WINDOW) - 

\  / 

\ _ / 

!  NO 

_ V _ 

UPDATE  ! 

TOTAL  ! 
SPIKES  ! 


_V_ 

/  \ 

/  \  NO 

<CRITERIA— 
\  MET  / 

\ _ / 

. > I  YES 

_ V _ 

SET  “BURST” 
PULSE 

COUNT  BURST 
SETUP  PARAMETER 
FOR  PRINT 


_V_ 

/  \ 

/FREE  \  NO 
<  REINF  >-- 
\  /  : 


SET  OVER¬ 
RIDE 

SET  COUNT 


/  NCRWND  \ 

:  <  — 

/ - \ 

_V_ 

/PRINT  &  DISPLAYV 

/  \ 

\  WINDOW  DATA  / 

/  \N 

\  / 

<  PRINT  > 

>  :  YES 


/  IREC  \  /  SETPLT  \ 

/ . . \  / . \ 

/WAIT  FOR  MESSAGE)-) (6)  FIGURE  6.3  (5)<— /PLOT  &  PRINTV 

\ _ /  CONTROL  PROGRAM:  RTCON  \ _ / 
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APPENDIX  A 
NCCP  MODULES 


CHEADR.FR 

CLCOM.FR 

CLIMOD.FR 

clrscr.fr 

concat.fr 

cplot.fr 

cplotcom.fr 

cplotp.fr 

cursor.fr 

D7PL0T.fr 

D7PRNT.FR 

dfault.fr 

DIODF.SR 

D  1 0 1  NT . SR 

EOTPRO.FR 

FSETUP.FR 

GVAL.FR 

GVAL 1 . FR 

HELPCO.FR 

HELPDR.FR 

IOSPLY.FR 

INITRC.FR 

KEYBRD.FR 

MOVMSG.FR 

MPLOT.FR 

NCCP . FR 

NCCPCOM.FR 

NCCPPAR.FR 

NCRHED.FR 

NCRPG.FR 

NCRSTS.FR 

NCRSUM.FR 

NCRWNO.FR 

OUTVAL.FR 

POSTP.FR 

PPLOT.FR 

PRTST1.FR 

PRTST2.FR 

RDFLD.FR 

REAL. SR 

RTCLI.FR 

RTCON.FR 

SELECT. FR 

SETP.SR 

SETPLT.FR 

STUPRT.FR 

WRFLD.FR 

ZTRANS.FR 


CRT  Header 

Common  used  by  Assembly  Language  Routines 

CLI  Value  Modification  Subroutine 

Clear  Screen 

Concatenate  Str ings 

CRT  Plot  -  Main  Routine  &  Control 

Common  Used  by  Plot  Routine 

CRT  Plotting  Rout i ne 

Position  Cur  sor 

Display  Window  Data  Subroutine 
Print  Window  Data  Subroutine 
SETUP  Default  Values 
Define  Discrete  I/O  Device 
Discrete  Interrupt  Routine 

End  of  Test  Processing  -  Write  Out  Data,  Get  Comments 

Generalized  Setup  Control  Program 

Input  a  Value  from  Keyboard,  Check  Limits 

Input  a  Value  from  Keyboard,  Check  Limits,  Check  Sign 

Command  Helps 

Drug  Helps 

Display  Certain  Setup  Parameters  on  Screen 

Initialize  Test  Variables 

Get  Two  Characters  from  the  Keyboard 

Move  a  Message  from  One  Array  to  Another  Array 

Mode  Change  Plot  Handler 

MAIN  PROGRAM  -  Controls  Program  Flow,  Calls  Plot, 
Writes  Data  to  Disk 

Common  Main 

System  Parameters 

Printer  Header 

Pr  i  nter  Page  Control 

Printer  Parameter  Status  Report 

Printer  Summary  Report  (Every  Print  Cycle) 

Printer  Wi ndow  Repor  t 

Output  a  Value  9  Specified  Position  on  CRT 

Setup  Post  Processor 

Print  Plot  Routine 

Pre-test  Overlay  *1  -  Main  Menu 

Pre-test  Overlay  #2  -  Open  Disk  File 

Set-up  Read  Field  Subroutine 

Real  Time  Clock  Control  Program 

Real  Time  CLI  Processor 

Real  Time  Control  Program  (Windows,  Histogram) 

Get  a  Value  for  Menu  Response 

Set-up  Table 

Set-up  Plot  Table 

Prints  Set-up  Values 

Set-up  Write  Field  Subroutine 

Converts  Numbers  to  ASCII  with  Leading  Zero's 
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APPENDIX  B 

NCCP  BUILD  PROCEDURE 


1.  Bring  up  RDOS  as  described  in  Section  3.1 

2.  Edit  .SR  Files  and  .FR  Files  as  required 

3.  a)  Assemble  .SR  Files  (MAC/L) 

b)  Compile  .FR  Files  (FORTRAN) 

c)  PRINT  Listing  Files  (.LS)  or  Source  Files  (.FR) 

d)  Delete  Listing  Files 

4.  Type: 

NCCPLD  (NL) 

If  SETP.SR  was  the  module  changed  then,  after  assembling  SETP, 
type: 

SETPLD  (NL) 

5.  When  the  above  step  is  done  (about  5  minutes),  type: 

PRINT  NCCP .MP  (NL) 

Save  the  printout,  this  is  your  Load  Map. 

A  printout  of  the  macro  command  NCCPLD  can  be  obtained  by 
typi ng: 

PRINT  NCCPLD. MC  (NL) 

It  should  be  noted  that  if  any  of  the  system  common  or  parameter 
definitions  are  modified,  it  might  be  necessary  to  recompile  all 
modules  that  use  common.  This  can  be  done  using  one  of  the  following 
MACRO'S: 

TOTALFOR  -  Compiles  all  programs 

NCCPFOR  -  Compiles  all  programs  that  use  NCCPCOM  Common 
CLCOMFOR  -  Compiles  all  programs  that  use  CLCOM  Common 

There  are  other  macro  commands  and  the  user  can  look  at  macro's 
ending  with  -LD.MC  to  determine  their  use. 
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APPENDIX  C 
OATA  FILE  STRUCTURE 


I  I  I  - 1  DATA  FILE  STRUCTURE: 

The  data  file  contains  several  different  types  of  data  records: 

*  Initial  Setup  Data 

*  A  Spike  History 

*  B  Spike  History  (when  defined) 

*  PLOT  Table 

*  Post  Test  Comments 


INITIAL  SETUP  DATA: 

The  first  record  in  the  file  is  the  initial  setup  record.  This  is  a 
fixed  length  record  of  SO  words  (200  bytes)  and  a  description  of  the 
record  is  contained  in  NCCPPAR.FR. 


SPIKE  HISTORY  DATA: 

The  spike  history  data  is  formated  in  1001  word  blocks  with  the 
first  word  of  each  block  indicating  the  type  of  record: 

1  *  A  Spikes/Buffer  0 

2  =  A  Spikes/Buffer  1 
3=8  Sp i kea/Buf for  0 
4  s  B  Spikes/Buffer  1 

Each  entry  in  the  table  is  a  pair  of  words  representing  the  time  of 
the  spike  in  milliseconds  relative  to  the  start  of  the  test.  The 
first  word  (bits  1-15)  contain  the  least  significant  bits  and  the 
second  word  (bits  1-15)  contain  the  most  significant  bits  of  the 
time.  If  the  first  word  of  a  pair  is  equal  to  -1,  then  that  is  the 
last  valid  entry  in  that  block. 


PLOT  TABLE  DATA: 

PLOTTBL,  the  buffer  which  contains  the  information  from  which  the 
plots  are  generated,  is  written  to  the  disk  periodically  throughout 
the  running  of  the  test  and  at  the  end  of  the  test.  A  plot  table 
record  is  identified  by  the  first  word  of  the  record  being  greater 
than  or  equal  to  six,  but  not  999.  This  number  times  2  plus  1 
equals  the  number  of  words  in  the  record.  The  format  of  the  data  is 
described  under  MI-2  "Plot  Table  Structure”  below. 


POST  TEST  COMMENTS: 

The  post  test  comments  are  written  in  ASCII  to  the  file  as  the  last 
record  in  the  file  and  their  record  type  is  999.  Eighty  characters 
are  written  even  though  the  user  is  restricted  to  72  characters  of 
i nput . 
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III- 


PLOT  TABLE  STRUCTURE: 

All  data  that  is  plotted,  including  "ON",  "OFF",  etc.,  is  stored  in 
the  Plot  Table,  "PLOTTBL".  The  1st  word  of  the  table  contains  the 
number  of  entries  in  the  table  and  this  is  immediately  followed  by 
the  1st  entry.  Each  entry  consists  of  two  words  defined  as  follows: 


First 

Word: 

*  of 

Criteria 

-1 

"ON"  Plot 

-2 

"OFF"  Plot 

-3 

Variable 

Change 

Second 

Word: 

*  of 

Spikes 

Os "ON"  Only 
not  0;  “ON" * 
Wi ndows/Fr ee 
Rei nforcement 

No  Meaning 

Bits  0-5  = 
Criteria 

Bits  6-15  = 

Reinforcement 

Duration 

The  table  is  sized  for  160  entries. 
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SPOKEN  PAPERS  BY  L.  STEIN 


DATE 

TITLE  AND  INSTITUTION 

LOCATION 

10/28/81 

"ENDORPHINS  AND  MEMORY  FORMATION" 

Opioids  in  Mental  Illness  Conference 

New  York, NY 

12/10/81 

"REINFORCEMENT  TRANSMITTERS" 

UCI  Biological  Chemistry  Seminar 

Irvine.CA 

2/1/82 

"OPERANT  CONDITIONING  AND  HIGHER 

MENTAL  FUNCTIONS",  UCI  Philosophy  of 

Brain  Meeting 

Irvine,CA 

3/23/82 

"CATECHOLAMINE  REINFORCEMENT  TRANS¬ 
MITTERS  AND  DEPRESSION",  Denghausen 
Depression  Meeting 

Martinique 

5/24/82 

"MECHANISMS  OF  POSITIVE  REINFORCEMENT" 

UCI  Psychobiology  Seminar 

Irvine ,CA 

6/29/82 

"MECHANISMS  OF  DRUG  ABUSE" 

ADAMA  Science  Press  Seminar  Series 

Washington,  D.C 

10/7/82 

"BEHAVIORAL  ACTIONS  OF  ENDORPHINS" 

DuPont  Central  Research  Seminar 

Wilmington,  DE 

11/9/82 

"PHARMACOLOGY  OF  BENZODIAZEPINES" 

Anaheim  Memorial  Hospital  Seminar 

Anaheim, CA 

3/25/83 

"OPERANT  CONDITIONING  OF  INDIVIDUAL 
NEURONS" .Society  of  Experimental 
Psychologists 

La  Jolla, CA 

4/6/83 

"POSITIVE  REINFORCEMENT  OF  NEURONS" 
Denghausen  Depression  Group,  New 

York  University 

New  York, NY 

5/10/83 

"REINFORCEMENT  MECHANISMS" 

Center  for  Neurobiology  of  Learning  & 
Memory  Seminar 

Irvine ,CA 

7/28/83 

"POSITIVE  REINFORCEMENT:  CELLULAR  OR 
SYSTEMS  PRIORITY",  Air  Force  Office 
of  Scientific  Research  Review 

Irvine, CA 

8/30/83 

"REWARD  SYSTEMS" 

Psychiatry  Department  Seminar,  UCI 

Medical  Center 

0range,CA 

Stein, L.  and  Belluzzi,J.D 


Final  Technical  Report  F49620-81-K-0015 


9/14/83  "MECHANISMS  OF  POSITIVE  REINFORCEMENT"  Orange, CA 

Neurology  Department  Seminar,  UCI 
Medical  Center 

12/1/83  "NEUROCHEMISTRY  OF  REWARD"  Irvine, CA 

UCI  Department  of  Microbiology  Seminar 

3/5/84  "POSITIVE  REINFORCEMENT  OF  SINGLE  Irvine, CA 

UNITS",  Center  for  Neurobiology  of 
Learning  &  Memory  Seminar 

3/15/84  CELLULAR  BASIS  OF  REWARD"  Chicago, IL 

Anhedonia  &  Affect  Deficit  States 

3/23/84  "CELLULAR  BASIS  OF  REWARD" 

Denghausen  Conference 

5/17/84  "CELLULAR  BASIS  OF  POSITIVE  REINFORCE-  Los  Angeles, CA 

MENT",  UCLA  Psychology  Depart.  Seminar 

6/9/84  "OPERANT  CONDITIONING  OF  SINGLE  NEURONS"  Boston.MA 

7th  Harvard  Symposium  on  Quantative 
Analysis  &  Behavior 
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11/8/83  "OPERANT  CONDITIONING:  CELLULAR  OR  Boston.MA 

SYSTEMS  PROPERTY",  Society  for  Neuro¬ 
science  Paper 

10/12/83  "PHARMACOLOGY  OF  LEARNING"  Orange, CA 

UCI  Department  of  Neurology  Seminar 
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